Content sharing by pointing to content

ABSTRACT

Integrating an application and a chat session includes a first user identifying content from the application, wherein the application is independent of any chat session, the first user initiating the chat session with at least a second user that is remote from the first user by transmitting the content from the first user to the second user, where the first and second user establish the chat session in response to the content being transmitted, and the first user and the second user sharing the content in connection with the chat session. The content may be a portion of a photograph, a portion of an image, a portion of text from a document, and/or a portion of a screen. The portion may be automatically identified following the user pointing and marking up the content. The first user may mark up the content before transmitting it to the second user.

RELATED APPLICATIONS

This application is a continuation of and claims priority to U.S. patentapplication Ser. No. 16/723,292, filed Dec. 20, 2019, entitled“Initializing Chat Sessions by Pointing to Content,” which is acontinuation of and claims priority to U.S. patent application Ser. No.14/261,618, filed Apr. 25, 2014 (now U.S. Pat. No. 10,523,454), entitled“Initializing Chat Sessions by Pointing to Content,” which claimspriority to U.S. Provisional Application No. 61/834,568, filed Jun. 13,2013, and entitled “Initializing Chat Sessions by Pointing to Content.”Each of these applications is incorporated herein by reference.

TECHNICAL FIELD

This application is directed to the field of transmission ofinformation, and more particularly to the field of preparing digitaldata for mobile communications.

BACKGROUND

Mobile chat has become a mainstream worldwide communication medium.According to market data, message volume for mobile chat has longexceeded a volume of mobile operator supported SMS (Short MessagingService) communications: for example, about 19 billion mobile chatmessages have been sent daily over mobile chat applications in 2012,compared with 17.5 billion SMS messages. A disparity between mobile chatapplications and traditional operator-supported messaging is expected toquickly grow; thus, volume forecasts for 2014 by market researchcompanies are in the range of 50 billion mobile chat messages vs. 21billion SMS messages per day. Some experts point out that the growthnumbers for mobile chat may be much higher if an acceptance of anindustry wide Joyn platform for unified messaging and file sharingacross mobile platforms and operators quickly progresses.

A cumulative number of users for the top eight mobile chat applicationshas exceeded 1.5 billion people in 2013. These applications are:WhatsApp (recently acquired by Facebook, Inc.), BlackBerry Messenger,Viber, Nimbuzz, Apple's iMessage, KakaoTalk, Facebook Messenger forAndroid, and TenCent QQ and WeChat. Mobile operators and smartphonemanufacturers are starting to add dedicated buttons to launch popularmobile chat applications with certain models of their phones, such as aWhatsApp button on a Nokia Asha 210 phone.

Wearable devices of different types, including Head-Mounted Displays(HMOs) and cameras, such as Google Glass, are expected to increasinglyinteroperate with mobile phones and applications, which may add newdimensions to mobile chat applications and pair them with augmentedreality and sensor data.

One key advantage of mobile chat applications that makes them adesirable communications medium for a mass user is associated with theirmultimedia sharing and platform integration capabilities. For example,each of the three leading mobile chat applications in Asia: WeChat inChina, KakaoTalk in Korea, and LINE in Japan are currently offeringemoji, stickers and mobile games directly integrated into chatplatforms; LINE has launched a line of cartoon characters with their ownshow and toys; KakaoTalk and Evernote Corporation of Redwood City,Calif. have announced integration of Evernote service features into theKakaoTalk chat platform. File and screen sharing and other enhancedcommunications features are already present in popular chat applicationslike Skype and have been included into short-term product roadmaps forother mobile chat applications.

Notwithstanding significant progress in the proliferation of mobile chatapplications and in the development of features of mobile chatapplications, mobile messaging still suffers from significantshortcomings with respect to capabilities of individual applications andthe role of the applications within an overall mobile softwareecosystem. One of the most noticeable flaws of mobile chat applicationsis an absence of meaningful integration of such applications intomainstream mobile software. Thus, in a widespread usage scenario, aperson needs to communicate with other chat participants in conjunctionwith current on-screen and application content: images, maps, documents,web pages, charts, video, etc. However, the use of mobile chatapplications is predominantly disconnected from such mainstream contentconsumption; it takes a significant effort and many steps to explain toother chat participants the nature of a user inquiry or a statementrelated to the present content on that user's screen. Additionally,existing methods for sharing such content are generally available onlyat the file exchange level and are time consuming and awkward.

Similarly, in an augmented reality context, a user may function withinreal life scenes where a communication need may be invited by a currentenvironment with its changing features and situations. The existingmobile chat applications are poorly suited to meet such real-timerequirements.

Accordingly, it is desirable to develop a cohesive interoperationbetween mainstream mobile applications and content management systems,on the one hand, and mobile chat applications, on the other hand.

SUMMARY

According to the system described herein, integrating an application anda chat session includes a first user identifying content from theapplication, wherein the application is independent of any chat session,the first user initiating the chat session with at least a second userthat is remote from the first user by transmitting the content from thefirst user to the second user, where the first and second user establishthe chat session in response to the content being transmitted, and thefirst user and the second user sharing the content in connection withthe chat session. The content may be a portion of a photograph, aportion of an image, a portion of text from a document, and/or a portionof a screen. The portion may be automatically identified following theuser pointing and marking up the content. The first user may mark up thecontent before transmitting it to the second user. The first user mayinitiate the chat with the content by a multi-touch gesture, a keyboardshortcut, and/or pointing an arrow drawn by software on a screen of adevice containing the content. One of the users may facilitate the chatsession using an augmented reality system and the content may be part ofa real scene viewed by the one of the users with the augmented realitysystem. The second user may be part of a list of participants defined bythe first user and/or part of a list of participants engaging in recentchat sessions with the first user. The second user may modify and markup the content. The second user may be prohibited from marking up thecontent and/or modifying the content. Prior to transmission of thecontent, the first user may select from a set of standard comments orquestions that are transmitted to the second user with the content. Thefirst user may confirm the content prior to transmission. The chatsession may run simultaneously on multiple devices of each of the users.The application may run on a non-mobile device of one of the users andthe chat session may run simultaneously on a mobile device of the one ofthe users. Content may be selected on the non-mobile device. The chatsession may be provided on a device using WhatsApp, BlackBerryMessenger, Viber, Nimbuzz, iMessage, KakaoTalk, Facebook Messenger,TenCent QQ, WeChat, or Skype. The chat session may be provided on adevice using a browser. The chat session may be a mobile chat session.The content may be provided on at least one mobile device.

According further to the system described herein, computer software,provided in a non-transitory computer-readable medium, integrates anapplication and a chat session. The software includes executable codethat identifies, for a first user, content from the application, wherethe application is independent of any chat session, executable code thatinitiates, for the first user, the chat session with at least a seconduser that is remote from the first user by transmitting the content fromthe first user to the second user, where the first and second userestablish the chat session in response to the content being transmitted,and executable code that shares the content between the first user andthe second user in connection with the chat session. The content may bea portion of a photograph, a portion of an image, a portion of text froma document, and/or a portion of a screen. The portion may beautomatically identified following the user pointing and marking up thecontent. The content may be marked up before being transmitted it to thesecond user. The chat may be initiated using the content by amulti-touch gesture, a keyboard shortcut, and/or pointing an arrow drawnby software on a screen of a device containing the content. An augmentedreality system may be used and the content may be part of a real sceneviewed by one of the users with the augmented reality system. The seconduser may be part of a list of participants defined by the first userand/or part of a list of participants engaging in recent chat sessionswith the first user. The second user may modify and mark up the content.The second user may be prohibited from marking up the content and/ormodifying the content. Prior to transmission of the content, the firstuser may select from a set of standard comments or questions that aretransmitted to the second user with the content. The first user mayconfirm the content prior to transmission. The chat session may runsimultaneously on multiple devices of each of the users. The applicationmay run on a non-mobile device of one of the users and the chat sessionmay run simultaneously on a mobile device of the one of the users.Content may be selected on the non-mobile device. The chat session maybe provided on a device using one of: WhatsApp, BlackBerry Messenger,Viber, Nimbuzz, iMessage, KakaoTalk, Facebook Messenger, TenCent QQ,WeChat, or Skype. The chat session may be provided on a device using abrowser. The chat session may be a mobile chat session. The content maybe provided on at least one mobile device.

The proposed system integrates chat applications into mobile and desktopsoftware workflow, including methods for selecting portions of contentfor transmission to chat participants, methods for identifying userinstructions to initiate chat session and transmit the selected content,including pointing to the content, and sharing of on-screen content withchat participants with a possible addition of user authored or standardcomments.

Integration of chat systems into software may include an ability toextract (clip) portions of on-screen content displayed to a user indesktop and mobile applications such as web browsers, document viewingand authoring applications (for example, office applications, image andvideo viewers and editors, photo capturing and editing software, games,etc.), and instantly initiate a mobile chat session where the extractedcontent is transmitted and shared between chat participants. Thisapproach cardinally differs from a non-transparent content transmissionmethod in the existing mobile chat applications, as explained elsewhereherein.

Depending on a platform, device, software title and interoperabilityfeatures, such content extraction may represent clipped content indifferent formats: it may be a snapshot of a selected portion of a userscreen or a whole screen in a common image format, with or without addeduser markup and/or caption; a portion of plain or formatted text of adocument; a photo taken in an original photo capturing or image editingapplication, with or without subsequent user markup; etc. Content mayalso be captured and transmitted in several alternative formats and thendisplayed to chat participants in one of the available formats,depending on participants' chat settings and capabilities of theirclient (browser based chat software).

Content selection may be done via native software selection means, suchas contiguous selection of textual fragments or a range of table cellsin office software; visual selection using graphical capabilitiesutilized in image editors or in markup software, such as EvernoteSkitch; etc. Additionally, selection software within an integrated chatsystem may be able to automatically choose a portion of content to whicha user has pointed as a natural selection candidate.

A chat session with a selected content may be initiated via a dedicatedgesture or a shortcut, such as pointing an arrow to a selected fragment,making a multi-touch gesture, entering a keyboard shortcut, etc. Suchuser instruction may result in an immediate opening of native mobilechat software or a browser based version of mobile chat software,starting a new chat session if the user is not in a session, andtransmitting the selected content to chat participants. The list ofparticipants may be defined on the fly via explicit user choice from thelist of contacts or may use various other options, such as indicatingparticipants of a most recent chat session as a default option orsuggesting most frequent participants of chat sessions with the user asa priority list. Specific display formats for the transmitted contentmay be different for different participants of the chat session, asexplained elsewhere herein. For some of the participants, thetransmitted content may be represented in an editable format, allowingthe participants to modify the original content and share the modifiedfragment. For other participants, markup on top of the received contentwithout modifying the transmitted fragment may be available in chatclient software used by the other participants. Yet another group ofparticipants may only be able to add comments to the received fragmentwithout any editing or markup features.

Prior to transmitting a selected fragment of content, the system maydisplay to sender a standard set of comments or questions to betransmitted with the content, such as “what is this?”, “how do I getthere?”, “which one?”, “look how great it is!”, etc. A chosen comment orquestion may be added to the transmitted content, possibly withadditional textual comments explicitly added by the sender. Users may beable to customize and augment lists of standard comments and questions.

When a user points to an object to initiate a chat session without anexplicit content selection, the system may determine a selectedcandidate and either highlight the selected candidate for approval bythe user prior to transmission or immediately transmit the selectedcandidate to chat participants. For example, a user may point a sessioninitialization arrow to a textual content and the system may offer inresponse a paragraph of text to which the user is pointing as a naturalselection candidate. In some embodiments, there may be additionalconventions regarding disposition of a pointing arrow with respect totext causing selection of different portions of text (in the absence ofan explicit content selection by the user). For example, pointing anarrow nearly vertically to a line of text may be interpreted as a lineselection, while pointing the same arrow nearly horizontally to a middleof a paragraph of text without touching the text may cause a selectionof the whole paragraph. Other examples may include images, drawings,charts or other objects within a document, pointing to which may causesuggesting of a target object as a selection candidate.

The system may run on multiple devices of a single user. For example,the user may be viewing or editing content on a tablet, a notebook or adesktop computer and may initiate a chat session from a smartphone bypointing to a portion of content on the tablet, notebook or desktop. Inthis case, the content may be first transmitted to the smartphone of theuser using any available network connection, such as Bluetooth, Wi-Fi,WAN (possibly using a cloud service to temporarily store and transmitcontent) or any combination of transmission mechanisms.

For augmented reality systems, such as Google Glass, the system presentsan opportunity to instantly point out an object or area of interest on aphotograph of a real scene the user is looking at, and transmit theobject or area of interest to other chat participants for answers,comments and markup by the other participants, which may direct furtheractions of the user.

BRIEF DESCRIPTION OF THE DRAWINGS

Embodiments of the system described herein will now be explained in moredetail in accordance with the figures of the drawings, which are brieflydescribed as follows.

FIG. 1 is a schematic illustration of initialization of a chat sessionby pointing to a selected or non-selected on-screen content, accordingto embodiments of the system described herein.

FIG. 2 schematically illustrates selection of a graphical content andstandard comments presented to a user prior to content transmission,according to embodiments of the system described herein.

FIG. 3 is a schematic illustration of two-way content transmissionduring a mobile chat session based on selection and pointing, accordingto embodiments of the system described herein.

FIG. 4 is a system flow diagram illustrating operation of the system,according to embodiments of the system described herein.

DETAILED DESCRIPTION

The system described herein provides a mechanism for initializing mobilechat sessions between participants instantly using and sharing contentpresent on a mobile device or a desktop computer of a session organizer,the system incorporates session initializing software into a commonapplication environment and a mobile chat application and implementsselection and pointing features to identify and transmit portions ofcontent.

FIG. 1 is a schematic illustration 100 of initialization of a chatsession by pointing to selected or non-selected on-screen content. Thechat session may use a browser on a mobile device and/or may bespecialized software such as WhatsApp, BlackBerry Messenger, Viber,Nimbuzz, Apple's iMessage, KakaoTalk, Facebook Messenger, TenCent QQ andWeChat. In other instances, an application such as Skype may be used. Atablet 110 is running a software application 120 with an applicationtoolbar 125. The system described herein is incorporated into theapplication environment and the system toolbar 130 with a pointing arrowtool 135 is displayed on the tablet screen.

FIG. 1 illustrates two scenarios of initialization of a mobile chatsession. Under the first scenario, a user selects a portion of text 140and draws a pointing arrow 150 to the selection. The pointing arrow 150causes instant interoperation of the tablet 110 with a smartphone 160 ofa user, so that a mobile chat application 170 on the smartphone 160 isinstantly launched if the user has not already been in a chat session.The selected content 140 is instantly transferred to the smartphone 160and placed it into a chat application window 180. The user may addcomments 185 to the original content and send a message with selectedcontent and comments to other chat participant(s). FIG. 1 schematicallyillustrates further communications where another participant of themobile chat session has received a message and sent back a reply message190.

Under the second scenario, the user is pointing with an arrow 152(drawing a pointing arrow) that points to an image 145 embedded in acurrently displayed document. This causes a selection marker 154 toappear around the image 145 indicating to the user which part of thedocument content is about to be transmitted to participants of a mobilechat session (a mobile chat portion on the smartphone 160 is not shownin FIG. 1 ).

FIG. 2 is a schematic illustration 200 of selection of graphical contentand of standard comments presented to a user prior to contenttransmission. A smartphone, shown on the left in a landscape mode 210 a,runs a photo capturing application 220; a photograph 225 of a cityskyline is displayed on a screen of the smartphone. The system describedherein is incorporated into an application environment and a toolbar ofthe smartphone with a pointing arrow 230 and a drawing tool 235 forobject selection is shown on the screen. The tool 235 allows selectingmultiple fragments of an on-screen content by freehand encircling thefragments. In FIG. 2 , the user encircles two fragments 240 a, 240 busing the tool 235 (possibly looking to clarify an address approaching afriend's location). After the user has drawn an arrow 250 pointing atone of the selected content fragments using a tool 230, the system maypresent the user with a list 260 of standard comments and/or questions.In FIG. 2 , the list 260 is arranged as a selectable menu and an itemchosen by the user, displayed over a black selection background,reflects a choice made by the user.

After initialization data is collected, the smartphone switches to amobile chat application shown in a smaller size and a portrait mode 210b. Then, the system captures a portion of the photograph 225 aroundselected fragments 240 a, 240 b and compiles a mobile chat message 270by combining a portion 280 of the photo 225 with a standard comment 290chosen by the user from the list 260. The message is ready for sharingwith other mobile chat participants, although the user may choose to addcustom textual comments.

FIG. 3 is a schematic illustration 300 of two-way content transmissionduring a mobile chat session based on selection and pointing. Asmartphone 310 runs a mobile chat application of a first participant320, which may send a mobile message 330, generated from user content onthe smartphone 310 using selection and pointing, as explained in moredetail in conjunction with FIG. 2 . The message 330 is received on adifferent smartphone 340 running a mobile chat application for a secondparticipant 350; note that there may be more participants of the mobilechat session. The second participant 350 also runs the system and atoolbar 360 a is displayed on a screen of the smartphone 340. Amagnified copy 360 b of the toolbar 360 a is shown in FIG. 3 forconvenience. The toolbar 360 a, 360 b may include, among other tools, acropping tool 362 to clip portions of received images; a text tool 364to add comments directly on images; and a pointing arrow tool 366. InFIG. 3 , the second participant 350 crops a received image to retain arelevant portion 372; adds a text comment 374 directly on the image,types in an additional text comment 376 in the chat window and uses apointing arrow tool 366 both to markup an image by adding an arrow 378and to send back a response.

Note that the behavior of the pointing arrow tool 366 may bepre-programmed or customized by the second participant 350 in a way thatis different from the embodiment of the system described in connectionwith FIG. 2 . In particular, the arrow 378 in FIG. 3 is used to send amobile message rather than to initiate object selection, as explainedelsewhere herein. Additionally, the pointing arrow tool 366 may bepre-programmed to stay as part of the image (the arrow 378) rather thandisappear; alternatively, the pointing arrow tool 366 may exhibitbehavior caused by additional user interface action, such as, forexample, pressing and holding the arrow 378 for a short time after thearrow 378 that has been drawn on the screen may cause its appearance onthe image as a permanent markup element.

Referring to FIG. 4 , a flow diagram 400 illustrates functioning of thesystem described herein. Processing starts at a step 410 where thesystem monitors user activities in an application environment on amobile device or on a desktop computer. After the step 410, processingproceeds to a test step 415, where it is determined whether the userperforms a pointing gesture. If not, then processing proceeds back tothe step 410. Otherwise, processing proceeds to a test step 420, whereit is determined whether the pointing gesture has been performed whenthere is already a content selected by the user utilizing system tools.If so, then processing proceeds to a test step 425, where it isdetermined whether the selection is a standard selection used in textediting and other applications that unambiguously identifies theselected object. If not, then processing proceeds to a step 430 wherethe system calculates selection area or objects, such as, for example, aportion 280 of an image containing multiple selections 240 a, 240 b inFIG. 2 .

If it is determined at the test step 420 that there is no contentalready selected, then processing proceeds to a step 435 where thesystem automatically identifies content for transmission using differentclues, which may be based on pointing arrow's direction and position, asexplained elsewhere herein. After the step 435, processing proceeds to astep 440, which may also be independently reached from the step 435 orfrom the step 425 if the user does not make a standard selection. At thestep 440, the system offers to the user a list of standard comments orquestions to choose from, as explained elsewhere herein, in particular,in the accompanying text to FIG. 2 . After the step 440, processingproceeds to a test step 445, where it is determined whether the user hasselected a standard comment from the list. If so, processing proceeds toa step 450, where the selected comment is added to the selected content,as explained in conjunction with FIG. 2 . After the step 450, processingproceeds to the step 455, which may be independently reached from thestep 445 if the user does not select a standard comment.

At the step 455, the system opens a mobile chat session. After the step455, processing proceeds to a step 460 where the system generates amessage, combining one or more of a selected content, a selectedstandard comment and a custom comment added by the user. After the step460, processing proceeds to the step 465 where the user adds oridentifies desired recipients of the mobile message using differenttechniques, as explained elsewhere herein. Note that in a situation whenthe user participates in an ongoing mobile chat session, rather thanopens a new session to present the selected content with comments, thestep 465 may be omitted, unless the user wants to alter the alreadyexisting list of participants in the context of continued mobile chatsession. After the step 465, processing proceeds to a step 470 where themobile chat application sends the message. After the step 470,processing is complete.

Various embodiments discussed herein may be combined with each other inappropriate combinations in connection with the system described herein.Additionally, in some instances, the order of steps in the flowcharts,flow diagrams and/or described flow processing may be modified, whereappropriate. Subsequently, elements and areas of screen described inscreen layouts may vary from the illustrations presented herein.Further, various aspects of the system described herein may beimplemented using software, hardware, a combination of software andhardware and/or other computer-implemented modules or devices having thedescribed features and performing the described functions. The mobiledevice may be a smartphone, although other devices, such as wearablecomputers, are also possible. The system described herein may beimplemented with any type of electronic screen capable of being actuatedby a touch screen, electromagnetic or other pen.

Note that the mobile device(s) may include software that is pre-loadedwith the device, installed from an app store, installed from a desktop(after possibly being pre-loaded thereon), installed from media such asa CD, DVD, etc., and/or downloaded from a Web site. The mobile devicemay use an operating system such as iOS, Android OS, Windows Phone OS,Blackberry OS and mobile versions of Linux OS.

Software implementations of the system described herein may includeexecutable code that is stored in a computer readable medium andexecuted by one or more processors, including one or more processors ofa desktop computer. The desktop computer may receive input from acapturing device that may be connected to, part of, or otherwise incommunication with the desktop computer. The desktop computer mayinclude software that is pre-loaded with the device, installed from anapp store, installed from media such as a CD, DVD, etc., and/ordownloaded from a Web site. The computer readable medium may benon-transitory and include a computer hard drive, ROM, RAM, flashmemory, portable computer storage media such as a CD-ROM, a DVD-ROM, aflash drive, an SD card and/or other drive with, for example, auniversal serial bus (USB) interface, and/or any other appropriatetangible or non-transitory computer readable medium or computer memoryon which executable code may be stored and executed by a processor. Thesystem described herein may be used in connection with any appropriateoperating system.

Other embodiments of the invention will be apparent to those skilled inthe art from a consideration of the specification or practice of theinvention disclosed herein. It is intended that the specification andexamples be considered as exemplary only, with the true scope and spiritof the invention being indicated by the following claims.

What is claimed is:
 1. A content sharing method, comprising: executing auser application at a first electronic device, including displayingcontent in a first interface of the user application and receiving auser selection of a subset of the content displayed in the firstinterface of the user application; automatically and in response to theuser selection of the subset of the content, transferring the subset ofthe content from the first electronic device to a second electronicdevice communicatively coupled to the first electronic device;displaying the subset of the content in a second interface of acommunication application, where the communication application isexecuted at the first and second electronic devices to facilitatecommunication between the first and second electronic devices;generating, in the communication application, a communication includingthe subset of the content; identifying a second user in a communicationsession for receiving the communication from a first user of the firstelectronic device; and transmitting, in the communication application,the communication including the subset of the content, wherein thecommunication is transmitted from the second electronic device to athird electronic device of the second user, thereby allowing the firstuser and the second user to share the content displayed on the firstelectronic device via the communication application.
 2. The method ofclaim 1, wherein the communication application includes a messagingapplication, and the communication includes a message.
 3. The method ofclaim 1, wherein the communication application is an instant messagingapplication, and the communication includes a real-time chat sessionimplemented by the instant messaging application.
 4. The method of claim1, further comprising: in response to the user selection of the subsetof the content, automatically launching the communication application atthe second electronic device; and receiving the communication in thecommunication application.
 5. The method of claim 1, wherein the seconduser is at least one of: part of a list of participants defined by thefirst user and part of a list of participants engaging in recentcommunication sessions with the first user, and the second user modifiesand marks up the subset of the content via the communicationapplication.
 6. The method of claim 1, wherein the first electronicdevice includes one of a tablet computer, a notebook computer and adesktop computer, and the second electronic device includes one of amobile phone.
 7. The method of claim 1, wherein the content includestext, and the subset of the content is a portion of the text.
 8. Themethod of claim 7, further comprising: displaying in the secondinterface of the communication application the subset of the content toallow a first user of the second electronic device to edit the subset ofthe content to be transmitted in the communication.
 9. The method ofclaim 7, wherein receiving the user selection of the subset of thecontent further comprises: receiving a user input of highlighting theportion of the text; detecting that a pointing affordance is associatedwith the user input; and in accordance with detection that the pointingaffordance is associated with the user input, identifying the userselection of the subset of the content.
 10. A content sharing system,comprising: one or more processors; and memory storing one or moreprograms to be executed by the one or more processors for: executing auser application at a first electronic device, including displayingcontent in a first interface of the user application and receiving auser selection of a subset of the content displayed in the firstinterface of the user application; automatically and in response to theuser selection of the subset of the content, transferring the subset ofthe content from the first electronic device to a second electronicdevice communicatively coupled to the first electronic device;displaying the subset of the content in a second interface of acommunication application, where the communication application isexecuted at the first and second electronic devices to facilitatecommunication between the first and second electronic devices;generating, in the communication application, a communication includingthe subset of the content; identifying a second user in a communicationsession for receiving the communication from a first user of the firstelectronic device; and transmitting, in the communication application,the communication including the subset of the content, wherein thecommunication is transmitted from the second electronic device to athird electronic device of the second user, thereby allowing the firstuser and the second user to share the content displayed on the firstelectronic device via the communication application.
 11. The contentsharing system of claim 10, wherein the content includes an imageembedded in a document associated with the content, and the subset ofthe content is an entirety or a fragment of the image.
 12. The contentsharing system of claim 11, wherein receiving the user selection of thesubset of the content further comprises: receiving a user input toassociate a pointing affordance with the image; and in accordance withdetection that the pointing affordance is associated with the userinput, identifying the user selection of the subset of the content. 13.The content sharing system of claim 12, wherein receiving the userselection of the subset of the content further comprises: receiving auser input to point and mark up the fragment of the image; and inaccordance with the user input, identifying the user selection of thefragment of the image as the subset of the content to be transmitted inthe communication.
 14. The content sharing system of claim 11, thememory further storing programs for: displaying a system toolbarconcurrently with the content in the first interface of the userapplication, the system toolbar including at least a drawing tool forselecting a fragment of an image and a pointing tool for selecting animage or a portion of text.
 15. A non-transitory computer readablestorage medium storing one or more programs, which when executed by acomputer system cause the computer system to implement operationscomprising: executing a user application at a first electronic device,including displaying content in a first interface of the userapplication and receiving a user selection of a subset of the contentdisplayed in the first interface of the user application; automaticallyand in response to the user selection of the subset of the content,transferring the subset of the content from the first electronic deviceto a second electronic device communicatively coupled to the firstelectronic device; displaying the subset of the content in a secondinterface of a communication application, where the communicationapplication is executed at the first and second electronic devices tofacilitate communication between the first and second electronicdevices; generating, in the communication application, a communicationincluding the subset of the content; identifying a second user in acommunication session for receiving the communication from a first userof the first electronic device; and transmitting, in the communicationapplication, the communication including the subset of the content,wherein the communication is transmitted from the second electronicdevice to a third electronic device of the second user, thereby allowingthe first user and the second user to share the content displayed on thefirst electronic device via the communication application.
 16. Thenon-transitory computer readable storage medium of claim 15, whereintransmitting the communication further comprises transmitting the subsetof the content from the first electronic device to a server system andstoring the subset of the content temporarily in the server system. 17.The non-transitory computer readable storage medium of claim 15, furthercomprising instructions for: receiving user comments in the secondinterface of the communication application, wherein the communicationincludes the user comments.
 18. The non-transitory computer readablestorage medium of claim 17, wherein the user comments are selected froma set of standard comments or questions for transmission with the subsetof the content.
 19. The non-transitory computer readable storage mediumof claim 15, wherein the user application is associated with anaugmented reality system, and the content is part of a real scene viewedby a user of the augmented reality system.
 20. The non-transitorycomputer readable storage medium of claim 15, wherein the communicationapplication includes a messaging application, and the communicationincludes a message.